[Serve] Optimize DeploymentDetails.deployment_route_prefix_not_set()
#46305
Conversation
Signed-off-by: Josh Karpel <[email protected]>
| raise ValueError( | ||
| "Unexpectedly found a deployment-level route_prefix in the " | ||
| f'deployment_config for deployment "{cls.name}". The route_prefix in ' | ||
| f'deployment_config for deployment "{v.name}". The route_prefix in ' |
There was a problem hiding this comment.
This was an unrelated bug - I suspect that either no one has ever hit this block, or they just accepted the AttributeError and figured out what was wrong some other way?
| @@ -947,10 +947,12 @@ class DeploymentDetails(BaseModel, extra=Extra.forbid, frozen=True): | |||
| def deployment_route_prefix_not_set(cls, v: DeploymentSchema): | |||
| # Route prefix should not be set at the deployment level. Deployment-level route | |||
| # prefix is outdated, there should be one route prefix per application | |||
| if "route_prefix" in v.dict(exclude_unset=True): | |||
| if ( | |||
| "route_prefix" in v.__fields_set__ | |||
There was a problem hiding this comment.
We need to support both pydantic v1 and v2. Could you import
from ray._private.pydantic_compat import IS_PYDANTIC_2
and also add support for pydantic 2?
There was a problem hiding this comment.
If I understand
ray/python/ray/_private/pydantic_compat.py
Lines 55 to 57 in 4ae8128
pydantic.v1 import to keep using the v1 API, so this call will be valid either way (I tested this code with Pydantic v2 installed and it worked as expected). But it would cause an issue when you switch to actually using Pydantic v2 models internally 🤔
... maybe this field will be fully deprecated by then and this validator will be deleted before the migration happens? 😁
There was a problem hiding this comment.
yea this should be fine because it's only touching our internal models
user models can be pydantic v2
| @@ -947,10 +947,12 @@ class DeploymentDetails(BaseModel, extra=Extra.forbid, frozen=True): | |||
| def deployment_route_prefix_not_set(cls, v: DeploymentSchema): | |||
| # Route prefix should not be set at the deployment level. Deployment-level route | |||
| # prefix is outdated, there should be one route prefix per application | |||
| if "route_prefix" in v.dict(exclude_unset=True): | |||
| if ( | |||
| "route_prefix" in v.__fields_set__ | |||
There was a problem hiding this comment.
yea this should be fine because it's only touching our internal models
user models can be pydantic v2
|
Thanks! |
Why are these changes needed?
We've identified this method as a hotspot when the Serve Controller is tracking a lot of apps, taking ~5% of total time at 3k apps. Serializing the whole object to check if a field is set is unnecessary in both Pydantic v1 and v2... if you're willing to use a dunder field in v1 :)
After this change, this check takes no significant time.
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.This behavior didn't seem to be tested anywhere already, and it looks like it's just protecting a deprecation, so I didn't add a unit test, but I checked that it does have the desired behavior manually: